@copilot what can we do about these limitations?
CLOB/NCLOB columns: prefetch is skipped; falls back to live query transparently.
LOB parameters (BLOB/CLOB as ? bind values): prefetch skipped — LOB references are session-scoped and cannot be reused on a separate prefetch connection.

Ideally we should be able to cache any type of data including LOB types. Can we not keep them in memory as well ? It might ok to skip queries that receive a LOB as parameter, but as a return type in a query we should be able to cache it.

@copilot what can we do about these limitations?
CLOB/NCLOB columns: prefetch is skipped; falls back to live query transparently.
LOB parameters (BLOB/CLOB as ? bind values): prefe...

Done in commit 563e3f8. CLOB/NCLOB (and LONGVARCHAR/LONGNVARCHAR) return columns are now fully cached — the character stream is read eagerly via getCharacterStream() / getNCharacterStream() and materialised as a String (identical to how BLOBs are materialised as byte[]). Readers are properly closed via try-with-resources.

The only remaining skip is for queries that pass LOB session references as input parameters (BLOB/CLOB ? bind values) — those are session-scoped objects that cannot be transferred to a separate prefetch connection.

@copilot some questions in relation to this implementation:
1 - Is cache isolated by datasource? meaning if to datasources share the same query they don't get the wrong cached data.
2 - Is there a job to clean up abandoned cached pages? We need a configuration that can set how long a cache can be held before being cleaned up, default it to 60 seconds.

@copilot some questions in relation to this implementation:
1 - Is cache isolated by datasource? meaning if to datasources share the same query they don't get the wrong cached data....

Both addressed in commit 77195b9:

1 - Cache isolation by datasource: Yes, it's now fully isolated. The cache key is datasourceId \u0001 normalizedSql — the SOH control character (\u0001) is used as separator since it cannot appear in SQL text or a connection hash, preventing any key collision. Two datasources running the same query get completely independent cache entries.

2 - Background cleanup job: A daemon ScheduledExecutorService thread now runs evictExpiredOrCompleted() on a configurable interval (default 60 seconds). The new config property is ojp.server.nextPageCache.cleanupIntervalSeconds (defaults to 60). The default TTL (ojp.server.nextPageCache.ttlSeconds) is also reduced from 300s to 60s to align with the recommended hold time. The cache implements AutoCloseable and exposes shutdown() to stop the scheduler cleanly.

@copilot make sure there is only one background job that checks all cached objects, there is no need for multiple threads cleaning.

@copilot create an integration test for Postgres with pagination. The test should create different amounts of records (99, 100, 101, 567(random), 1000 - make sure you use parameterized tests, not duplications) in a given table that contains multiple types and at least one LOB type. There should be tests paginating 100 records per page. You have to run a separate OJP server in the main CI workflow on a different port with pagination cache enabled and point the new integration test at this new server. You can use the same postgres db already present in the workflow, just name the table uniquely. Make sure all records are returned correctly when using cache and pagination, and assert every value. If a lob makes sure all bytes are returned correctly.
Let me know of your questions, concerns, suggestions, and opinions.

@copilot make sure there is only one background job that checks all cached objects, there is no need for multiple threads cleaning.

Done in commit 268cdaa. The ScheduledExecutorService is now a static final field (CLEANUP_EXECUTOR) shared across all NextPagePrefetchCache instances — exactly one background thread exists per JVM regardless of how many instances are created. Each instance holds only an AtomicReference<ScheduledFuture<?>> to track and cancel its own eviction task independently without touching the shared executor or affecting other instances. shutdown() uses AtomicReference.getAndSet(null) to atomically swap the task handle out, avoiding concurrent double-cancel races.

@copilot create an integration test for Postgres with pagination. The test should create different amounts of records (99, 100, 101, 567(random), 1000 - make sure you use parameterized tests, not dupl...

Done in commit e953c30.

New test class — PostgresPaginationCacheIntegrationTest:

• Parameterized over 5 record counts: 99, 100, 101, 567, 1000 (via postgres_prefetch_cache_connections_with_record_counts.csv), covering partial-page, exact-page, one-over, non-round, and 10-page scenarios
• Table ojp_pfx_pg_{recordCount} has 7 columns of varied types: INT, BIGINT, BOOLEAN, VARCHAR(100), TEXT, and BYTEA (LOB)
• Rows are inserted with fully deterministic per-row values via a PreparedStatement batch
• Paginates with LIMIT 100 OFFSET ? (ORDER BY id) and asserts every column value on every row, including a byte-exact comparison of the BYTEA column — handles both byte[] and PostgreSQL hex-escape string representations (\xHH…)
• Enabled only with -DenablePostgresPrefetchCacheTests=true; skips automatically otherwise

CI workflow — postgres-test job:

• Starts a third OJP server container ojp-server-prefetch-cache on port 10594 with ojp.server.nextPageCache.enabled=true, ttlSeconds=60, prefetchWaitTimeoutMs=5000
• Passes -DenablePostgresPrefetchCacheTests=true in the existing "Test (ojp-jdbc-driver) with PostgreSQL enabled" step
• Logs from the new container are captured in the always-run log step

Concern/note: PostgreSQL does not have a native BLOB/CLOB type — its binary LOB is BYTEA. The prefetch cache materialises BYTEA as byte[] (via rs.getBytes() for Types.BINARY/VARBINARY), which is then serialised as ByteString over gRPC and returned to the client as byte[]. The test verifies this round-trip byte-for-byte.

@copilot why do we need any configuration on the server?

You're right — there's no need for server-side per-datasource configuration for enabled. The server owns the global toggle (ojp.server.nextPageCache.enabled); whether a specific datasource in a client application uses the cache is the client's concern.

Fixed in 37a84bf. The per-datasource enabled flag is now a client-side connection property:

# ojp.properties — client application
# Disable the prefetch cache for the "random-access" datasource
random-access.ojp.nextPageCache.enabled=false

The server reads ojp.nextPageCache.enabled from the ConnectionDetails properties sent on connect() and registers it per connection. When the property is absent, the global server flag applies as the default. The ojp.server.nextPageCache.datasource.<name>.enabled server property and its tests have been removed.

@copilot implement similar pagination integration tests for all other databases supported.

@copilot implement similar pagination integration tests for all other databases supported.

Done in 065b101. Pagination cache integration tests now cover all supported databases:

Each test paginates over 5 record counts (99, 100, 101, 567, 1000), asserts every column including byte-exact binary comparison, and is disabled by default (enabled via enable{DB}PrefetchCacheTests=true). The CI workflow was updated to start a second OJP server with the prefetch cache enabled (port 10594) for each database job.

@copilot

Tests failing

MySQL and MariaDB

Error: Errors:
Error: MySQLMariaDBPaginationCacheIntegrationTest.testMariaDBPaginationWithPrefetchCache:91->runPaginationTest:116->assertPage:212 » NumberFormat For input string: "false"
Error: MySQLMariaDBPaginationCacheIntegrationTest.testMariaDBPaginationWithPrefetchCache:91->runPaginationTest:116->assertPage:212 » NumberFormat For input string: "false"
Error: MySQLMariaDBPaginationCacheIntegrationTest.testMariaDBPaginationWithPrefetchCache:91->runPaginationTest:116->assertPage:212 » NumberFormat For input string: "false"
Error: MySQLMariaDBPaginationCacheIntegrationTest.testMariaDBPaginationWithPrefetchCache:91->runPaginationTest:116->assertPage:212 » NumberFormat For input string: "false"
Error: MySQLMariaDBPaginationCacheIntegrationTest.testMariaDBPaginationWithPrefetchCache:91->runPaginationTest:116->assertPage:212 » NumberFormat For input string: "false"

SQL server

logs have errors like this
[main] ERROR org.openjproxy.jdbc.Driver - Failed to establish connection
java.sql.SQLException: Failed to create datasource: Failed to create HikariCP DataSource: Failed to initialize pool: The TCP/IP connection to the host localhost, port 1433 has failed. Error: "Connection refused. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
at org.openjproxy.grpc.client.GrpcExceptionHandler.handle(GrpcExceptionHandler.java:32)
at org.openjproxy.grpc.client.StatementServiceGrpcClient.connect(StatementServiceGrpcClient.java:64)
at org.openjproxy.jdbc.Driver.connect(Driver.java:111)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)

tests are failing

Error: Errors:
Error: SQLServerPaginationCacheIntegrationTest.testPaginationWithPrefetchCache:109->assertPage:228 » NumberFormat For input string: "false"
Error: SQLServerPaginationCacheIntegrationTest.testPaginationWithPrefetchCache:109->assertPage:228 » NumberFormat For input string: "false"
Error: SQLServerPaginationCacheIntegrationTest.testPaginationWithPrefetchCache:109->assertPage:228 » NumberFormat For input string: "false"
Error: SQLServerPaginationCacheIntegrationTest.testPaginationWithPrefetchCache:109->assertPage:228 » NumberFormat For input string: "false"
Error: SQLServerPaginationCacheIntegrationTest.testPaginationWithPrefetchCache:109->assertPage:228 » NumberFormat For input string: "false"

@copilot

Tests failing
...

Fixed in d7cf13c.

MySQL/MariaDB: MariaDB's TINYINT(1) is mapped to Java Boolean by the MariaDB Connector/J driver. When read via rs.getInt("val_bool"), the client received the string "false" and threw NumberFormatException. Fixed by switching to rs.getBoolean("val_bool") with comparison id % 2 == 0 — consistent with the PostgreSQL, H2, and CockroachDB tests that already used getBoolean().

SQL Server: SQL Server's BIT type is also mapped to Boolean by the MSSQL JDBC driver, so rs.getInt("val_bool") would fail the same way. Applied the same fix (getBoolean() + id % 2 == 0).

Update the documentation as per now the configurations are in the client side instead of the server. Make sure that is the case on the integration tests.

Fixed in 4ba9354:

Root cause of the gap: DatasourcePropertiesLoader only forwarded ojp.connection.pool.* and ojp.xa.* keys to the server — ojp.nextPageCache.* properties were silently dropped, so the client-side enabled flag was never actually sent.

Changes:

• DatasourcePropertiesLoader: added OJP_NEXT_PAGE_CACHE_PREFIX = "ojp.nextPageCache." so all ojp.nextPageCache.* properties (including ojp.nextPageCache.enabled and per-datasource variants like reporting.ojp.nextPageCache.enabled) are now forwarded to the server at connect time.
• Test ojp.properties: added ojp.nextPageCache.enabled=true so the integration tests explicitly opt in via the client-side property.
• All 7 pagination cache integration test Javadocs updated to describe both the server-side global flag (ojp.server.nextPageCache.enabled=true) and the client-side per-datasource flag (ojp.nextPageCache.enabled=true in ojp.properties).
• Docs (NEXT_PAGE_PREFETCH_CACHE.md, ojp-server-configuration.md, part2-chapter6-server-configuration.md): split config reference tables into server-side and client-side sections; expanded examples to show both opt-in and opt-out per datasource.

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud